Accurate geometry reconstruction of vascular structures using implicit splines

3-D visualization of blood vessel from standard medical datasets (e.g. CT or MRI) play an important role in many clinical situations, including the diagnosis of vessel stenosis, virtual angioscopy, vascular surgery planning and computer aided vascular surgery. However, unlike other human organs, the vasculature system is a very complex network of vessel, which makes it a very challenging task to perform its 3-D visualization. Conventional techniques of medical volume data visualization are in general not well-suited for the above-mentioned tasks. This problem can be solved by reconstructing vascular geometry. Although various methods have been proposed for reconstructing vascular structures, most of these approaches are model-based, and are usually too ideal to correctly represent the actual variation presented by the cross-sections of a vascular structure. In addition, the underlying shape is usually expressed as polygonal meshes or in parametric forms, which is very inconvenient for implementing ramification of branching. As a result, the reconstructed geometries are not suitable for computer aided diagnosis and computer guided minimally invasive vascular surgery. In this research, we develop a set of techniques associated with the geometry reconstruction of vasculatures, including segmentation, modelling, reconstruction, exploration and rendering of vascular structures. The reconstructed geometry can not only help to greatly enhance the visual quality of 3-D vascular structures, but also provide an actual geometric representation of vasculatures, which can provide various benefits. The key findings of this research are as follows: 1. A localized hybrid level-set method of segmentation has been developed to extract the vascular structures from 3-D medical datasets. 2. A skeleton-based implicit modelling technique has been proposed and applied to the reconstruction of vasculatures, which can achieve an accurate geometric reconstruction of the vascular structures as implicit surfaces in an analytical form. 3. An accelerating technique using modern GPU (Graphics Processing Unit) is devised and applied to rendering the implicitly represented vasculatures. 4. The implicitly modelled vasculature is investigated for the application of virtual angioscopy

3D vasculature segmentation using localized hybrid level-set method

Background: Intensity inhomogeneity occurs in many medical images, especially in vessel images. Overcoming the difficulty due to image inhomogeneity is crucial for the segmentation of vessel image. Methods: This paper proposes a localized hybrid level-set method for the segmentation of 3D vessel image. The proposed method integrates both local region information and boundary information for vessel segmentation, which is essential for the accurate extraction of tiny vessel structures. The local intensity information is firstly embedded into a region-based contour model, and then incorporated into the level-set formulation of the geodesic active contour model. Compared with the preset global threshold based method, the use of automatically calculated local thresholds enables the extraction of the local image information, which is essential for the segmentation of vessel images. Results: Experiments carried out on the segmentation of 3D vessel images demonstrate the strengths of using locally specified dynamic thresholds in our level-set method. Furthermore, both qualitative comparison and quantitative validations have been performed to evaluate the effectiveness of our proposed model. Conclusions: Experimental results and validations demonstrate that our proposed model can achieve more promising segmentation results than the original hybrid method does

High precision implicit modeling for patient-specific coronary arteries

High precision geometric reconstruction of patient-specific coronary arteries plays a crucial role in visual diagnosis, treatment decision-making, and the evaluation of the therapeutic effect of interventions in coronary artery diseases. It is also a fundamental task and a basic requirement in the numerical simulation of coronary blood flow dynamics. In this paper, a new implicit modeling technique for the geometric reconstruction of patient-specific coronary arteries has been developed. In the proposed method, the coronary arteries geometry is reconstructed segment by segment using radial basis functions with ellipsoid constraint from the point cloud obtained with a volumetric vascular image segmentation method, and the individually reconstructed coronary branches are then combined using a shape-preserving implicit blending operation to form a complete coronary artery surface. The experiment results and validations indicate that the reconstructed vascular shapes are of high smoothness and faithfulness

ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation

Most recent scribble-supervised segmentation methods commonly adopt a CNN framework with an encoder-decoder architecture. Despite its multiple benefits, this framework generally can only capture small-range feature dependency for the convolutional layer with the local receptive field, which makes it difficult to learn global shape information from the limited information provided by scribble annotations. To address this issue, this paper proposes a new CNN-Transformer hybrid solution for scribble-supervised medical image segmentation called ScribFormer. The proposed ScribFormer model has a triple-branch structure, i.e., the hybrid of a CNN branch, a Transformer branch, and an attention-guided class activation map (ACAM) branch. Specifically, the CNN branch collaborates with the Transformer branch to fuse the local features learned from CNN with the global representations obtained from Transformer, which can effectively overcome limitations of existing scribble-supervised segmentation methods. Furthermore, the ACAM branch assists in unifying the shallow convolution features and the deep convolution features to improve model’s performance further. Extensive experiments on two public datasets and one private dataset show that our ScribFormer has superior performance over the state-of-the-art scribble-supervised segmentation methods, and achieves even better results than the fully-supervised segmentation methods. The code is released at https://github.com/HUANGLIZI/ScribFormer

Accurate geometry modeling of vasculatures using implicit fitting with 2D radial basis functions

Accurate vascular geometry modeling is an essential task in computer assisted vascular surgery and therapy. This paper presents a vessel cross-section based implicit vascular modeling technique, which represents a vascular surface as a set of locally fitted implicit surfaces. In the proposed method, a cross-section based technique is employed to extract from each cross-section of the vascular surface a set of points, which are then fitted with an implicit curve represented as 2D radial basis functions. All these implicitly represented cross-section curves are then being considered as 3D cylindrical objects and combined together using a certain partial shape-preserving spline to build a complete vessel branch; different vessel branches are then blended using a extended smooth maximum function to construct the complete vascular tree. Experimental results show that the proposed method can correctly represent the morphology and topology of vascular structures with high level of smoothness. Both qualitative comparison with other methods and quantitative validations to the proposed method have been performed to verify the accuracy and smoothness of the generated vascular geometric models

LViT: Language meets Vision Transformer in Medical Image Segmentation

Deep learning has been widely used in medical image segmentation and other aspects. However, the performance of existing medical image segmentation models has been limited by the challenge of obtaining sufficient high-quality labeled data due to the prohibitive data annotation cost. To alleviate this limitation, we propose a new text-augmented medical image segmentation model LViT (Language meets Vision Transformer). In our LViT model, medical text annotation is incorporated to compensate for the quality deficiency in image data. In addition, the text information can guide to generate pseudo labels of improved quality in the semi-supervised learning. We also propose an Exponential Pseudo label Iteration mechanism (EPI) to help the Pixel-Level Attention Module (PLAM) preserve local image features in semi-supervised LViT setting. In our model, LV (Language-Vision) loss is designed to supervise the training of unlabeled images using text information directly. For evaluation, we construct three multimodal medical segmentation datasets (image + text) containing X-rays and CT images. Experimental results show that our proposed LViT has superior segmentation performance in both fully-supervised and semi-supervised setting. The code and datasets are available at https://github.com/HUANGLIZI/LViT.Comment: Accepted by IEEE Transactions on Medical Imaging (TMI

LViT: Language meets Vision Transformer in Medical Image Segmentation

Deep learning has been widely used in medical image segmentation and other aspects. However, the performance of existing medical image segmentation models has been limited by the challenge of obtaining sufficient high-quality labeled data due to the prohibitive data annotation cost. To alleviate this limitation, we propose a new text-augmented medical image segmentation model LViT (Language meets Vision Transformer). In our LViT model, medical text annotation is incorporated to compensate for the quality deficiency in image data. In addition, the text information can guide to generate pseudo labels of improved quality in the semi-supervised learning. We also propose an Exponential Pseudo label Iteration mechanism (EPI) to help the Pixel-Level Attention Module (PLAM) preserve local image features in semi-supervised LViT setting. In our model, LV (Language-Vision) loss is designed to supervise the training of unlabeled images using text information directly. For evaluation, we construct three multimodal medical segmentation datasets (image + text) containing X-rays and CT images. Experimental results show that our proposed LViT has superior segmentation performance in both fully-supervised and semi-supervised setting. The code and datasets are available at https://github.com/HUANGLIZI/LViT

Towards additive manufacturing oriented geometric modeling using implicit functions

Surface-based geometric modeling has many advantages in terms of visualization and traditional subtractive manufacturing using computer-numerical-control cutting-machine tools. However, it is not an ideal solution for additive manufacturing because to digitally print a surface-represented geometric object using a certain additive manufacturing technology, the object has to be converted into a solid representation. However, converting a known surface-based geometric representation into a printable representation is essentially a redesign process, and this is especially the case, when its interior material structure needs to be considered. To specify a 3D geometric object that is ready to be digitally manufactured, its representation has to be in a certain volumetric form. In this research, we show how some of the difficulties experienced in additive manufacturing can be easily solved by using implicitly represented geometric objects. Like surface-based geometric representation is subtractive manufacturing-friendly, implicitly described geometric objects are additive manufacturing-friendly: implicit shapes are 3D printing ready. The implicit geometric representation allows to combine a geometric shape, material colors, an interior material structure, and other required attributes in one single description as a set of implicit functions, and no conversion is needed. In addition, as implicit objects are typically specified procedurally, very little data is used in their specifications, which makes them particularly useful for design and visualization with modern cloud-based mobile devices, which usually do not have very big storage spaces. Finally, implicit modeling is a design procedure that is parallel computing-friendly, as the design of a complex geometric object can be divided into a set of simple shape-designing tasks, owing to the availability of shape-preserving implicit blending operations